Questions Administered by Telephone or In Person: Differences in Interviewer-Respondent Interactions

When choosing a mode for data collection of computer-assisted surveys, a researcher has three main options available: the computer-assisted telephone interview (CATI), the computer-assisted personal interview (CAPI) or a web interview (i.e., a self-adminstered interview). Generally, CAPI allows for collecting most complex data, of the highest quality, but only when interviewers are well-trained and effort is made to monitor and manage interviewers during field work. This higher data quality in CAPI interviews may be due to the finding that presence of an interviewer reduces the amount of respondents’ satisficing behaviors (i.e., not investing the required effort to provide meaningful answers, see Heerwegh 2008). An interviewer can motivate respondents, presumably by means of rapport (Garbarski et al 2016) to invest the amount of cognitive effort required to answer questions thoughtfully. However, with interviewers administering the survey their social presence may also give rise to respondents’ reflection of social consequences of providing information to a relative stranger. By looking at response distributions it has been shown that social desirability bias and satisficing are more prevalent in CATI than in CAPI (see Holbrook et al 2003), and lowest in (self-administered) Web interviews. To fully explain such effects, it makes sense to study actual behaviors in interviewer-respondent interactions. In this approach, generally referred to as interaction coding or behavior coding (see Ongena and Dijkstra, 2006), interactions are systematically evaluated on deviations of the so-called paradigmatic sequence. A paradigmatic sequence is the interaction as intended by the researcher, with a sequence consisting of only two or three actions: the interviewer reading the question exactly as worded, the respondent providing an answer that exactly matches one of the response options, and optionally, an interviewer acknowledgement. Any deviation from this sequence may indicate problems in the questionnaire or the interviewing procedure. However, to our knowledge, CATI and CAPI interactions have never been systematically compared. By analyzing interactions of 60 CATI and 54 CAPI interviews that originated from a mixed-mode experiment using the European Social Survey questionnaire (Haan 2015), we found mixed differences with respect to behaviors in CATI and CAPI interactions. For example, interviewer laughter appeared to be more common in CATI than in CAPI, but apologetic utterances such as ‘sorry’ occurred equally often in both modes. Furthermore, a significant difference was found in the number of words uttered. Question-answer sequences contained more words in CATI than in CAPI. This is partly explained by the fact that for many questions in the CAPI survey show cards were used. Lack of show cards in CATI extends interactions due to less efficient communication about response alternatives. Further analysis showed that respondents in CATI had more difficulty in formulating their response than in CAPI. These task-related issues may contribute to the effect of decreased trust and motivation of respondents in CATI interviews, and may subsequently explain the increased level of satisficing and social desirability bias in this survey mode compared to CAPI

Chapter 12: Differences in interaction quantity and conversational flow in CAPI and CATI interviews. Appendix 12

Appendix 12A Differences in Length of Interviewer-Respondent Interactions in CAPI and CATI Interviews Table A12A.1 Response Rrates (AAPOR RR1) Table A12A.2 Descriptive Statistics for Questions and Respondent Characteristics Table A12A.3 Means and Correlations Trimmed number of Turns and Events in CAPI and CATI by Question Characteristics Table A12A.4 Means and Correlations Trimmed number of Turns and Events by Question Characteristic

Mode matters:Effects of survey modes on participation and answering behavior

In survey research it is important to have a good representation of the population. However, it appears that young adults, full-time workers, inhabitants of large cities and ethnic minorities are difficult to survey. This research has studied how these groups can be reached better. In an experiment, potential survey participants (respondents) could choose how they wanted to participate: by a web survey, a personal interview or a telephone interview. Respondents who chose the web viewed pre-recorded clips of an interviewer reading the questions to them for half of the survey (video-web survey). The experiment showed that young adults and full-time workers have a preference for web surveys. So offering these groups this method in future surveys could improve their representation. However, survey methods such as web or telephone can also affect the way respondents answer questions. Respondents can invest minimal effort when participating or can adapt their answers to social norms (socially desirable responding). The experiment showed that the degree of socially desirable responding and investing effort in video-web was equivalent to the web survey without video. The answers of both web surveys differed from the answers given in the personal and telephone interviews. From this it can be concluded that adding videos in a web survey does not improve data quality for web surveys, but does also not deteriorate data quality. Socially desirable responding is further investigated conducting three experiments. These experiments provided more insights about how the sensitivity of questions for socially desirable responding can be established

Can we predict device use? An investigation into mobile device use in surveys

In this study, we investigate whether mobile device use in surveys can be predicted. We aim to identify possible motives for device use and build a model by drawing on theory from technology acceptance research and survey research. We then test this model with a Structural Equation Modeling approach using data of seven waves of the GESIS panel. We test whether our theoretical model fits the data by focusing on measures of fit, and by studying the standardized effects of the model. Results reveal that intention to use a particular device can predict actual use quite well. Ease of smartphone use is the most meaningful variable: if people use a smartphone for specific tasks, their intention to use a smartphone for survey completion is also more likely. In conclusion, investing in ease of use of mobile survey completion could encourage respondents to use mobile devices. This can foremost be established by building well-designed surveys for mobile devices

Reaching Hard-to-Survey Populations: Mode Choice and Mode Preference

This study assesses the effect of response-mode choices on response rates, and responsemode preferences of hard-to-survey populations: young adults, full-time workers, big city inhabitants, and non-Western immigrants. Using address-based sampling, a stratified sample of 3,496 households was selected. The first group of sample members was contacted face to face and could choose between a CAPI and web response mode. The second group, contacted by telephone, could choose between CATI and web. The third group, contacted by telephone, was randomly allocated to a response mode. Our address-based sampling technique was successful in reaching most of the hard-to-survey groups. Insufficient numbers of non- Western immigrants were reached; therefore this group was excluded from our analyses. In our mixed-effect models, no significant effects on the willingness to participate were found for mode choice. We found that full-time workers and young adults were significantly more likely to choose web over CAPI when contacted face to face

Shaking hands in a busy waiting room:The effects of the surveyor’s introduction and people present in the waiting room on the response rate

Although waiting room surveys are frequently conducted, methodological studies on this topic are scarce. Behaviour of surveyors in waiting rooms can easily be controlled, and these surveys also allow for collection of paradata; relevant information on the circumstances of a request to participate in survey research. In this paper, we present the results of an experiment systematically manipulating surveyor’s handshakes and verbal introduction of their names. Patients scheduled for radiological examinations were approached to take part in a survey. An observer noted circumstances in the waiting room (CT or MRI) such as the number of people present.In the CT waiting room, willingness to participate was higher when no other people were filling out the survey than when there were other people filling out the survey. Thus, scarcity effects seemed to play a major role in the decision to participate. In addition, a patient waiting alone was more likely to fully complete the questionnaire, than patients accompanied by one or more caregivers. There was no effect of the surveyor’s handshake or verbal name introduction on survey participation, which is a fortunate outcome in light of social distances measures fighting COVID-19